Introduce IO::Event::WorkerPool for blocking_operation_wait.#139
Merged
samuel-williams-shopify merged 13 commits intoJun 11, 2025
Merged
Conversation
samuel-williams-shopify
force-pushed
the
native-work-pool
branch
from
0c79172 to
2911f07
Compare
samuel-williams-shopify
force-pushed
the
native-work-pool
branch
from
2911f07 to
f9164a2
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR adds a thread‐based IO::Event::WorkerPool and integrates it into a test fiber scheduler via Ruby’s blocking_operation_wait hook, enabling non‐blocking execution of GVL‐releasing operations.
- Introduces
IO::Event::WorkerPoolC extension and registers it inInit_IO_Event - Adds
IO::Event::TestSchedulerfixture to delegate blocking operations to the worker pool - Adds tests for pool construction, stats, and integration with large
IO::Buffer.copyoperations
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| test/io/event/worker_pool.rb | New tests for pool instantiation, stats API, and scheduler integration |
| fixtures/io/event/test_scheduler.rb | Defines IO::Event::TestScheduler to route blocking ops to WorkerPool |
| ext/io/event/worker_pool.h | Declares the Init_IO_Event_WorkerPool initializer |
| ext/io/event/fiber.c | Changed visibility of IO_Event_Fiber_current from static to global |
| ext/io/event/event.c | Calls Init_IO_Event_WorkerPool to register the extension |
| ext/extconf.rb | Adds worker_pool.c to the build list and feature detection for blocking hooks |
Comments suppressed due to low confidence (4)
test/io/event/worker_pool.rb:11
- [nitpick] Consider adding unit tests for
WorkerPool#callto verify that submitted tasks execute and that the returned promise resolves or rejects correctly.
describe IO::Event::WorkerPool do
ext/extconf.rb:26
- The build list includes "io/event/worker_pool.c" but that source file isn't committed. This will break compilation; ensure the .c implementation is present or remove it from
$srcs.
$srcs = ["io/event/event.c", "io/event/time.c", "io/event/fiber.c", "io/event/worker_pool.c", "io/event/selector/selector.c"]
test/io/event/worker_pool.rb:7
- The test requires
io/event/test_scheduler, but the fixture file lives underfixtures/.... Consider usingrequire_relativeor adjusting$LOAD_PATHso the test can load the scheduler definition.
require "io/event/test_scheduler"
ext/io/event/fiber.c:38
- [nitpick]
IO_Event_Fiber_currentwas made a global symbol by removingstatic. Unless it needs external linkage, restorestaticto limit symbol exposure.
VALUE IO_Event_Fiber_current(void) {
76ea5c7 to
3e3075a
Compare
samuel-williams-shopify
force-pushed
the
native-work-pool
branch
from
33e6b0b to
791b4ed
Compare
samuel-williams-shopify
merged commit 0345ded
into
socketry:main
36 of 40 checks passed
|
This is very exciting to see! 👀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This implements a thread-based worker pool that integrates with Ruby's experimental fiber scheduler
blocking_operation_waithook to handle blocking operations asynchronously. The WorkerPool uses feature detection to support the newrb_fiber_scheduler_blocking_operation_*functions introduced in recent Ruby versions, providing a promise-based interface for submitting work to a thread pool without blocking the event loop. For example, when blocking operations like largeIO::Buffer.copycalls (>1MiB) release the GVL, fiber schedulers can delegate them to the worker pool, allowing the main thread to continue processing other fibers.Types of Changes
Contribution